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FOREWORD 


The  U.S.  Army  Research  Institute  for  the  Behavioral  and 
Social  Sciences  ( ARI)  has  been  actively  involved  in  the 
development  of  After  Action  Review  (AAR)  methods  and  tools  for 
the  live,  virtual  and  constructive  training  environments 
beginning  with  the  development  of  tactical  engagement  simulation 
in  the  mid-seventies.  Many  of  the  findings  from  this  work  have 
been  incorporated  within  the  Standard  Army  AAR  System  (STAARS) 
concept  which  includes  the  proposed  standardization  of  AAR 
products  across  training  environments  for  each  unit  type  and 
echelon. 

This  report  describes  the  functions  AAR  aids  can  serve, 
differences  among  functions  served  by  various  types  of  AAR  aids, 
and  the  benefits  and  costs  of  standardizing  AAR  aids  across 
training  environments .  This  report  also  clarifies  the  concept  of 
standardized  AAR  aids,  and  it  describes  a  technique  for  defining 
a  standard  set  of  AAR  aids  for  a  particular  unit  type  or  echelon. 

The  work  described  in  this  report  is  a  portion  of  research 
task  2114,  SYNTRAIN:  Distributed  Interactive  Simulation  Systems. 
This  tasks  supports  a  Memorandum  of  Agreement  entitled  "Training 
Research  Support  of  Combined  Arms  Tactical  Trainer  Development 
Efforts,"  signed  24  Feb  93.  Parties  to  this  agreement  are  the 
U.S.  Army  Project  Manager  for  Combined  Arms  Tactical  Trainer  and 
ARI.  This  work  is  also  a  portion  of  research  task  2137,  Training 
Analysis  and  Feedback  Aids  (TAAF-Aids )  for  Live  Environments. 

This  task  supports  a  Memorandum  of  Record  signed  by  the  TRADOC 
Combat  Training  Support  Directorate  and  ARI. 


ZITA  M.  SIMUTIS 
Technical  Director 
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A  GUIDE  TO  STANDARDIZING  AFTER  ACTION  REVIEW  (AAR)  AIDS 

Introduction 

The  After  Action  Review  Process 


The  After  Action  Review  (AAR)  is  the  U.S.  Army's  approach  for 
providing  feedback  to  units  after  collective  training  exercises 
(Department  of  the  Army,  1993).  The  objectives  of  the  AAR  are  to 
decide  what  happened,  why  it  happened,  and  how  to  improve  or 
sustain  performance. 

Unlike  a  critique,  the  AAR  is  an  interactive  process 
in  which  exercise  participants  discuss  mission  planning  and 
execution  under  the  guidance  of  a  trainer  (Scott,  1983) . 

The  starting  point  for  the  AAR  is  normally  a  description  of 
the  unit's  plans  for  the  mission  followed  by  a  discussion  of 
what  happened  during  the  mission. 

The  discussions  can  be  guided  in  part,  through  the  use  of 
information  displays  illustrating  what  happened  during  an 
exercise.  These  displays  may  simply  serve  the  purpose  of 
refreshing  the  memory  of  exercise  participants,  or  they  may 
provide  a  new  perspective  on  exercise  events.  For  example, 
the  graph  shown  in  Figure  1  shows  when  each  vehicle  in  a 
tank  platoon  fired  and  the  result  of  each  firing  event  (0 
indicates  a  miss,  X  indicates  a  hit,  and  K  indicates  a 
catastrophic  kill) .  This  particular  figure  shows  that  one  tank 
did  not  participate  in  the  engagement,  another  tank  fired  many 
rounds  without  a  single  hit,  and  a  third  tank  had  many  hits  but 
only  one  kill.  The  information  in  the  figure  can  differ  markedly 
from  the  group's  collective  perception  of  what  happened  during 
the  exercise. 


A14  00  00  0  0  00  K=Kill 

H=Hit 

A13  K  0  0=Miss 

A12 

All  XKX  X  XX  X 


250  260  270  280  290 

SECONDS  ELAPSED  SINCE  START  OF  EXERCISE 
Figure  1.  Timing  and  effects  of  tank  main  gun  fires,  by  vehicle. 
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Application  Across  Live,  Virtual  and  Constructive 
Environments 


The  AAR  process  is  intended  to  apply  to  live,  virtual, 
constructive,  and  mixed  environment  exercises.  A  live  exercise 
is  one  in  which  operational  equipment  and  actual  terrain  is  used, 
such  as  when  a  platoon  maneuvers  in  its  tanks .  Virtual  exercises 
involve  the  networking  of  simulators  to  make  it  possible  for 
crews  to  interact  together  on  a  common  terrain  database. 
Information  produced  by  each  simulator,  such  as  its  location  on 
the  terrain  database,  is  transmitted  over  a  network  and  picked  up 
by  other  simulators.  The  graphics  generator  for  each  simulator 
employs  network  data  and  data  from  a  common  terrain  database  to 
provide  a  current  "out  the  window"  view  of  the  world  for  crew 
members  (Thorpe/  1987) .  Constructive  simulations  represent  units 
as  an  aggregate  without  simulating  each  entity  within  a  unit. 
(Stober,  Kraus,  Foss,  Franceschini ,  and  Petty,  1995),  and  this 
environment  has  been  used  largely  to  support  command  and  staff 
training.  A  synthetic  theater  of  war  (STOW)  environment  contains 
a  mix  of  at  least  two  types  of  environments  (Sottilare,  1995) . 

Benefits  of  Standardizing  AAR  Aids 


Many  benefits  may  be  gained  by  standardizing  AAR  aids  for  a 
particular  echelon  and  unit  type.  The  greatest  benefits  are 
gained  when  the  aids  are  standardized  across-  training 
environments.  Standardizing  AAR  aids  for  a  particular  group  does 
not  mean  that  a  specific  set  of  aids  will  be  produced  for  an 
exercise  regardless  of  what  happens  during  the  exercise.  Instead 
it  means  that  there  is  a  set  of  predefined  AAR  aids  that  a 
trainer  may  select  among  to  fit  the  outcome  of  a  specific 
exercise.  For  example,  armor  platoons  may  use  a  data  table  with 
a  specific  format  to  assess  or  illustrate  the  quality  of  fire 
control  during  defensive  missions.  If  poor  fire  control  or 
exceptionally  effective  fire  control  contributed  to  the  outcome 
of  the  exercise,  a  trainer  may  use  the  data  table  during  the  AAR. 
If  the  trainer  wants  to  focus  on  other  points  during  the  AAR, 
there  is  no  requirement  to  use  the  data  table.  The  benefits  of 
AAR  aid  standardization  are  discussed  below. 

Reduces  total  resources  required  to  produce  an  AAR .  When 
the  proponent  for  a  particular  unit  type  and  echelon  has  decided 
upon  the  aids  and  aid  features  useful  to  that  group,  it  becomes 
possible  to  write  software  that  relieves  many  tedious  tasks  from 
the  shoulders  of  trainers.  Such  software  may  even  include  the 
application  of  artificial  intelligence  to  automatically  reduce 
the  number  of  candidate  AAR  aids  to  those  likely  to  be  of 
interest,  given  a  specific  outcome. 
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Provides  the  potential  to  reduce  software  development  costs. 
From  1994  through  1997,  at  the  request  of  the  Project  Manager  for 
Combined  Arms  Tactical  Trainer  (PM-CATT) ,  ARI  hosted  an  annual 
AAR  Conference.  One  of  the  key  reasons  for  this  conference  was 
to  alert  proponents  for  AAR  systems  to  the  duplications  in  the 
capabilities  of  existing  AAR  systems.  The  effort  to  develop  a 
standardized  set  of  AAR  aids  within  each  unit  type  and  echelons 
can  provide  the  information  needed  by  engineers  to  decide  whether 
it  is  feasible  to  design  one  or  two  AAR  systems  that  can  meet  the 
needs  of  all  proponents. 

Links  all  collective  training  exercises  with  capstone  events. 
A  major  benefit  of  standardizing  displays  across  environments  is 
that  it  links  exercises  with  capstone  training  environments,  such 
as  rotations  to  a  combat  training  center  (CTC) .  If  a  CTC 
employed  a  particular  data  table  to  show  how  well  a  particular 
unit  performed  resupply  operations,  units  would  want  to  see  the 
same  table  after  participating  in  live,  virtual,  or  live 
exercises  at  home  station.  Performance  feedback  at  CTCs  has  a 
high  degree  of  credibility,  and  we  want  to  standardize  AAR  aids 
in  manner  that  capitalizes  on  this  credibility. 

Ensures  aids  are  readily  interpretable.  Standardization  is 
also  expected  to  help  ensure  that  AAR  aids  will  be  immediately 
interpretable  to  exercise  participants.  If  many  minutes  are 
required  to  explain  what  an  aid  means  then  it  will  be  of  little 
or  no  value  in  the  training  environment.  Standardizing  of  AAR 
aids  within  echelons  and  across  environments  helps  to  make  sure 
that  exercise  participants  will  be  familiar  with  the  aids  used  in 
AAR  sessions. 

Can  help  ensure  training  is  task-based.  To  the  extent  that 
AAR  aids  assess  the  outcome  of  task  performance,  they  also  ensure 
there  is  a  task-oriented  focus  to  the  AAR.  For  example,  the . 
standard  for  the  armor  platoon  task  "execute  a  column  formation" 
is  "the  platoon  executes  the  column  formation  without  delay  and 
without  stopping  movement."  AAR  aids  that  illustrate  a  unit  took 
too  long  to  change  to  the  column  formation,  came  to  a  stop  when 
changing  formations,  or  moved  in  a  formation  that  does  not  fit 
the  definition  of  a  good  column  formation  help  to  focus  the  AAR 
on  task  execution. 

Purpose  of  Guide 

The  Standardized  Army  After  Action  Review  System  (STAARS) 
program  includes  the  goal  of  providing  a  standard  set  of  AAR 
products  for  specific  unit  types  and  echelons  that  can  be  applied 
across  the  L/V/C  and  STOW  environments  (Department  of  the  Army, 
1996).  This  document  provides  guidance  for  implementing  this 
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goal.  In  addition  to  describing  the  benefits  of  standardizing 
AAR  aids,  this  guide: 


o  describes  general  types  of  AAR  aids  and  selected  variations  in 
g  the  way  the  various  types  have  been  implemented; 
o  clarifies  the  concept  of  AAR  aid  standardization; 
o  discusses  the  utility  of  each  type  of  aid; 

o  presents  a  technique  for  defining  a  standardized  set  of  AAR 
aids  for  a  specific  unit  type  and  echelon;  and 

o  describes  the  importance  of  having  an  AAR  editing  capability  to 
reduce  potential  problems  in  standardizing  AAR  aids. 
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Types  of  AAR  Aids 


Ground  Truth  Displays 


Ground  truth  displays  illustrate  actual  exercise  events. 

This  section  of  the  document  provides  a  description  of  general 
types  of  AAR  aids.  More  detailed  descriptions  of  specific 
implementations  of  various  types  can  be  reviewed  to  see  how  the 
design  features  of  the  aid  (the  specific  way  in  which  the  aid 
type  is  implemented)  may  have  to  be  modified  to  assist  in 
examining  specific  aspects  of  performance  (National  Simulation 
Center,  1997b;  Fernan  and  Dryer,  1994;  Meliza,  Bessemer, 

Burnside,  and  Shlechter,  1992;  Meliza  and  Tan,  1996;  Shlechter, 
Bessemer,  Rowatt,  and  Nesselroade,  1994)  .  For  example,  an  early 
version  of  an  AAR  system  included  the  ability  to  replay  vehicle 
movement  and  firing  events  on  a  moment  by  moment  basis  over  a 
grid  map  without  terrain  features.  This  type  of  aid  (called  a 
two  dimensional  animated  replay)  worked  well  for  examining 
certain  aspects  of  performance,  but  examining  other  aspects 
required  information  about  the  terrain  situation.  Therefore,  the 
AAR  system  was  modified  to  allow  the  two  dimensional  animated 
plan  view  to  "show"  or  "hide"  terrain  features  at  the  option  of 
the  user.  An  important  part  of  the  job  of  standardizing  AAR  aids 
is  identifying  the  design  features  that  need  to  included  for  each 
general  type  of  aid. 

In  many  cases,  more  than  one  type  of  display  can  be  used  to 
illustrate  the  same  event.  In  general,  the  best  display  to  use 
for  a  particular  purpose  is  the  one  that  makes  a  point  most 
efficiently  at  a  specific  point  in  time.  Efficiency  is 
determined,  in  part,  by  inherent  differences  among  aids  in  terms 
of  the  time  required  to  illustrate  a  point  (e.g.,  a  table  showing 
how  many  times  each  vehicle  fires  during  a  fifteen  minute . 
engagement  can  quickly  make  the  point  that  only  a  few  vehicles 
were  involved  in  the  engagement,  vice  watching  a  replay  and. 
trying  to  keep  track  of  which  vehicles  fired) .  Efficiency  is 
also  determined  by  the  overall  mix  of  AAR  aids  employed.  If  a 
trainer  can  make  a  series  of  points  using  a  single  type  of  AAR 
aid,  this  may  be  preferable  to  jumping  from  one  type  of  aid  to 
another.  An  important  part  of  the  job  of  standardizing  AAR  aids 
is  considering  the  speed  with  which  type  of  aids  can  be  used  to 
make  specific  points.  Another  important  part  of  the  job  is  to 
consider  the  variety  of  aids  that  can  be  used  to  make  a 
particular  point  to  provide  trainers  with  the  capability 
to  use  the  aids  that  are  most  effective  in  a  particular  session. 
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Two  dimensional  (2D)  animated  plan  view  replay.  This  aid 
shows  a  2D  replay  of  exercise  events  over  a  map  display  with  unit 
control  measures  marked.  Icons  may  be  used  to  represent 
individual  entities  (usually  vehicles)  or  they  may  represent  a 
unit  (e.g.,  platoon).  At  a  minimum,  these  displays  show  movement 
of  entities  or  units.  They  may  also  show  indirect  fire  impacts, 
firing  of  individual  vehicles,  the  status  of  individual  vehicles 
(alive,  damaged,  or  destroyed) ,  vehicle  orientation,  and  gun  tube 
orientation.  Animated  plan  view  replays  are  commonly  used  in  the 
live,  virtual  and  constructive  environments. 

The  2D  replay  may  also  provide  the  synchronized  replay  of 
tactical  communications.  It  may  also  show  when  line-of-sight 
exists  among  friendly  and  enemy  forces  and  when  entities  are 
employing  protective  measures  (e.g.,  aircraft  using  flares). 

This  list  of  information  that  can  be  shown  in  2D  displays  is  not 
exhaustive.  Additional  types  of  information  may  need  to  be 
included  to  examine  particular  aspects  of  unit  behavior. 

2D  replays  can  be  conducted  at  normal  speeds  (one  minute  of 
replay  covers  one  minute  of  exercise  time)  or  at  speeds  much 
greater  than  normal  (e.g.,  one  minute  of  replay  covers  six 
minutes  of  exercise  time) .  Replays  can  also  be  paused  to  allow 
viewers  to  study  an  activity  in  greater  detail.  Many  replay 
systems  also  provide  the  capability  for  users  to  jump  directly 
forward  or  backward  from  one  point  in  time  to  another . 

Snapshots .  Snapshots  fall  into  two  general  types.  One  type 
of  snapshot  simply  captures  the  contents  of  an  animated  2D 
display  at  a  specific  point  in  time.  Figure  2  is  an  example  of 
such  a  display,  and  it  shows  the  movement  formation  and 
dispersion  of  a  platoon  at  the  time  it  was  first  engaged  by  the 
enemy.  The  other  type  of  snapshot  may  provide  information  not 
contained  in  the  animated  2D.  Because  the  snapshot  is  a  static 
display,  additional  information  can  be  added  without  cluttering 
the  display.  Figures  3  and  4  illustrate  how  new  types  of 
information  may  be  added  to  a  snapshot  to  support  training 
points.  In  this  case,  one  tank  from  a  platoon  engages  the  enemy 
for  several  minutes  without  assistance  from  the  remainder  of  the 
platoon.  The  snapshots  help  to  explain  why  the  rest  of  the 
platoon  was  not  involved.  These  figures  indicate  where  line-of- 
sight  (LOS)  exists  between  vehicles  immediately  before  Tank  D 
engages  the  enemy  (a  solid  line  indicates  LOS,  while  a  dashed 
line  indicates  non-LOS) .  Figure  3  shows  that  Tank  D  has  LOS  with 
three  of  the  four  enemy  tanks,  but  does  not  have  LOS  with  any  of 
the  other  vehicles  in  its  platoon.  Figure  4  shows  that  Tanks  A 
and  B  do  not  have  LOS  with  any  of  the  enemy  vehicles. 
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Figure  4 .  Snapshot  showing  lack  of  LOS  among  tanks  A  and  B  and 
enemy  vehicles. 


Battle  flow.  The  battle  flow  display  is  another  version  of 
the  2D  display.  A  battle  flow  display  or  "snail  trail ^  provides 
a  trace  of  the  movement  of  individual  vehicles  or  a  unit.  The 
trace  is  provided  over  a  map  display,  and  locations  of  vehicles 
may  be  marked  at  specific  intervals  (e.g.,  every  five  minute). 
The  major  use  of  a  battle  flow  display  is  to  illustrate  problems 
in  navigation  such  as  wandering  and  backtracking .  The  advantage 
of  a  battle  flow  display  over  an  animated  plan  view  is  that  one 
"picture"  can  immediately  show  a  navigation  problem  rather  than 
having  a  unit  watch  a  replay  for  several  minutes .  Figure  5  is  a 
battle  flow  display  tracing  the  movement  of  three  tanks  from  one 
battle  position  to  another  over  a  period  of  roughly  fifteen 
minutes . 
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At  one  point  it  was  believed  that  battle  flow  displays 
provided  a  means  of  assessing  unit  formations  during  movement 
(e.g.,  do  move  abreast  throughout  an  assault,  or  are  vehicles 
adequately  dispersed  during  a  lengthy  movement?) .  Unfortunately 
using  battle  flow  displays  for  these  applications  requires  that 
position  updates  be  provided  by  each  vehicle  at  exactly  the  same 
time,  which  is  rarely  the  case.  Further  certain  applications 
require  update  rates  that  are  more  frequent  than  can  be  supported 
by  simulations  or  communications  bandwidths . 


Start  Title:  1515SO  Battle  Flow  Final  Tine:  152953 

R  <PLT  2>  :  B22  B  cPLI  2>  :  B23 

C  <PLT  2/XERD>  :  B21  D  <PLT  2-/SERG>  -  B24 


Figure  5.  Battle  flow  display  tracing  paths  of  three  tanks 
withdrawing  to  an  alternate  battle  position. 

Fire  fight  displays.  The  fire  fight  display  is  another 
version  of  the  2D  display.  The  purpose  of  the  fire  fight  display 
is  to  show  where  direct  and  indirect  fires  impact  over  a  period 
of  time  selectable  by  the  user.  Instead  of  individual  round 
impacts  being  briefly  shown  as  they  are  with  the  2D  replay,  the 
fire  fight  display  aggregates  impacts  over  time.  If  one  wants  to 
see  where  fires  are  and  are  not  being  placed,  the  fire  fight 
display  can  provide  this  information  immediately.  At  a  glance,  a 
unit  can  see,  for  example,  whether  they  fired'  only  within  their 
assigned  sector.  Figure  6  provides  an  example  of  a  fire  fight 
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display  in  which  direct  fires  are  shown  by  shot  lines  showing  the 
location  of  firing  entities  and  the  location  of  round  impacts. 

One  early  implementation  of  the  fire  fight  attempted  to 
employ  color  codes  to  convey  information  about  the  results  of 
each  direct  fire  event  (Meliza  and  Tan,  1996) .  This  approach  did 
not  work  well,  because  one  shot  line  would  tend  to  hide  another. 


Fira  Fight  S  t  Air  t  Tine:  150300 

Fire  Tine:  15X200 
End  Tine:  151200 


Figure  6.  Fire  fight  display  showing  location  of  firer  and 
impacts  of  rounds  over  time. 

Range  fans .  Range  fans  are  static  2D  displays  that  show 
areas  that  could  be  covered  by  a  unit ' s  direct  and  indirect 
fires.  This  type  cf  aid  is  most  appropriate  for  a  defensive 
mission.  A  number  of  examples  of  range  fans  are  contained  in  a 
report  by  Fernan  and  Dryer  (1994) . 

Video  clips.  In  all  environments,  video  clips  may  be  used  to 
capture  face  to  face  interactions  among  personnel  (e.g.,  issuing 
an  operations  order),  interactions  of  individuals  with  machines, 
or  the  contents  of  computer  screens  as  a  function  of  time.  The 
last  two  applications  of  video  clips  have  become  more  important 
with  digitization  of  the  battlefield  and  the  advent  of  the 
Applique  and  other  visually-based  digital  communications  systems. 

The  current  major  use  of  video  clips  is  in  the  live 
environment  where  videos  can  be  made  to  show  what  a  unit's 
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actions  looked  like  from  the  enemy's  perspective,  what  actions  a 
unit  performed  while  preparing  to  execute  a  mission,  and  what  the 
threat  situation  looked  like  from  a  unit's  perspective.  Used  in 
this  way,  the  video  clip  serves  substantially  the  same  function 
as  the  three  dimensional  view  described  below  does  in  virtual 
exercises . 

The  use  of  video  clips  for  AAR  sessions  present  certain 
logistical  problems.  The  first  problem  concerns  scheduling  the 
creation  of  a  video.  The  trainer  must  anticipate  a  significant 
event  and  make  arrangements  to  have  personnel  available  at  the 
appropriate  site  and  time  to  create  the  video.  The  second 
problem  concerns  editing  the  video  so  that  it  can  be  used  to  make 
a  point  without  exercise  participants  having  to  wait  while  a 
trainer  fast  forwards  through  a  video  to  reach  an  event  of 
interest . 

Three  dimensional  (3D)  animated  replays.  The  3D  animated 
replay,  like  the  video  clip,  is  an  out  the window  view  of  an 
exercise  that  may  be  employed  in  the  live,  virtual,  or 
constructive  environments.  The  3D  replay  uses  an  electronic  data 
stream  (data  on  movement  of  entities  or  units,  and  data  on  firing 
events)  to  create  a  virtual  view  of  an  exercise.  This  replay 
capability  can  be  used  in  any  environment  in  which  an  electronic 
data  stream  is  available  to  implement  the  3D  view. 

In  effect,  the  3D  animated  replay  covers  only  a  portion  of 
the  activities  that  can  be  shown  using  video  clips.  For  example, 
the  3D  replay  is  not  appropriate  for  showing  interactions  among 
humans  and  man/machine  interactions  (unless  the  training 
environment  creates  virtual  views  of  humans) .  An  advantage  of 
the  3D  view  is  that  it  allows  an  "out  the  window"  view  of  the 
battlespace  to  be  created  instantaneously  from  any  point/ 
perspective  in  the  battlespace.  The  3D  view  is  often  referred  to 
as  the  "stealth"  view,  because  a  trainer  can  observe  the  action 
from  any  perspective  during  an  exercise  without  being  seen  by 
participants . 

Tables  and  graphs .  These  displays  can  be  used  in  all 
environments.  The  data  used  to  prepare  these  displays  may  come 
from  instrumented  systems  that  automatically  collect  exercise 
data,  and/or  they  may  be  based  upon  observations  of  behavior  made 
by  trainers  or  data  collectors.  Figure  7  provides  an  example  of 
a  graph  showing  the  number  of  rounds  fired  by  four  platoon 
vehicles  as  a  function  of  time.  In  this  case  the  platoon  as  a 
whole  was  involved  in  two  different  engagements  over  a  thirty 
minute  span.  Three  platoon  vehicles  participated  in  the  first 
engagement,  and  two  participated  in  the  second.  One  of  the 
platoon  vehicles  did  not  fire  during  either  engagement.  During 
the  first  engagement,  one  of  the  vehicles  engaged  the  enemy  for 
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five  minutes  without  assistance  from  the  remainder  of  the 
platoon. 


Rounds  Fired  ouer  Tine  by  Vehicles  in  Platoon 
< In  1  ninute  intervals) 


Figure  7 .  Graph  showing  contributions  of  each  of  four  tanks  to 
engagements  during  two  periods  of  contact  with  enemy  forces. 


For  certain  aspects  of  unit  performance,  graphs  and  tables 
provide  the  fastest  way  to  illustrate  a  point.  For  example, 
consider  a  case  where  a  trainer  wants  to  make  the  point  that  only 
two  of  the  four  vehicles  in  a  platoon  were  involved  in  an 
engagement  that  lasted  ten  minutes.  The  trainer  may  fast-forward 
through  a  replay  of  the  engagement,  leaving  it  to  exercise 
participants  to  individually  keep  track  of  who  is  firing  and  who 
is  not.  Alternatively  the  trainer  may  use  a  table  providing  the 
number  of  rounds  fired  by  bumper  number,  showing  that  only  two 
vehicles  fired. 

A  weakness  of  tables  and  graphs  is  that  they  do  not  provide 
information  about  terrain,  friendly  situation,  and  enemy 
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situation  variables  shown  in  other  types  of  displays.  Additional 
information  is  often  needed  to  help  interpret  the  information  in 
tables  and  figures  and  to  support  follow-on  discussions  necessary 
to  diagnose  the  cause  of  a  problem. 

Exercise  timelines.  Exercise  timelines  show  when  events 
occur  during  an  exercise.  They  may  be  based  upon  data  that  are 
automatically  collected  electronically,  and/or  they  may  be  based 
upon  behavioral  observations .  A  commonly  used  timeline  based 
upon  behavioral  observations  is  one  showing  when  leaders  at 
various  echelons  issued  their  orders  relative  to  the  time  a  unit 
executed  its  mission.  It  is  used  in  deciding  whether  units . 
follow  the  guidance  to  take  no  more  than  one-third  of  the  time 
between  receipt  of  their  orders  and  mission  start  time  to  develop 
and  issue  orders  for  the  next  echelon. 

Behavioral  checklists .  AAR  aids  may  also  include  displays 
that  show  whether  a  unit  met  standards  described  in  doctrine,  but 
a  trainer  taking  this  approach  needs  to  be  prepared  to  defend 
ratings.  To  the  extent  that  the  ratings  are  based  upon, 
observations  of  the  occurrence/non-occurrence  of  specific  events 
(e.g.,  the  leader  did  or  did  not  conduct  a  backbrief  after 
issuing  an  order) ,  defending  a  rating  is  less  likely  to  be  a 
problem.  Further,  if  the  checklist  is  used  to  help  a  unit 
diagnose  a  problem  that  has  already  been  identified,  one  can 
expect  the  unit  to  be  more  accepting  of  ratings. 

One  should  also  keep  in  mind  that  a  checklist  can  also  be 
completed  by  a  unit  during  the  AAR.  For  example,  a  trainer  may 
show  a  checklist  that  has  not  been  completed  to  a  unit  and  ask 
the  unit  which  standards  they  feel  they  met.  A  trainer  may  also 
put  up  a  blank  piece  of  paper  and  ask  a  unit  to  make  a  list 
(e.g.,  "if  you  were  to  repeat  this  mission,  what  would  you  do 
differently  the  next  time?) " 

"How  to  Fight"  and  "How  to  Train"  Guidance 

Displays  can  also  be  used  to  support  AARs  by  describing 
alternative  courses  of  action  and  by  describing  training 
strategies  that  can  be  used  to  improve  or  sustain  performance. 
These  displays  may  be  important  in  helping  a  unit  decide  why  a 
performance  problem  occurred  and  in  identifying  potential 
corrective  actions . 

These  displays  may  take  the  form  of  text  or  figures  from  "how 
to  fight"  guidance  describing  tactical  doctrine  for  performing  a 
mission  or  task.  Many  examples  of  "how  to  fight"  displays  are 
provided  in  Appendix  J  of  Brown  et  al .  (1998) . 


13 


"How  to  fight"  guidance  may  even  take  the  form  of  ground 
truth  data  from  past  exercises.  For  example,  if  a  unit  had 
trouble  placing  direct  and  indirect  fires  on  the  enemy,  a  trainer 
might  show  a  fire  fight  display  or  animated  2D  replay  from  an 
exercise  in  which  a  unit  was  able  to  quickly  mass  fires  on  enemy 
positions. 
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The  Standard  Army  After  Action  Review  System  (STAARS) 
Concept  Regarding  Standardized  AAR  Aids 

The  Operational  Requirements  Document  (ORD)  for  the  Standard 
Army  AAR  System  (STAARS)  identifies  the  need  to  "provide  high 
quality  standardized  AAR  products  appropriate  to  the  echelon (s) 
being  trained  or  analyzed"  (National  Simulation  Center,  1997a) . 
STAARS  is  intended  to  be  applied  across  the  live,  virtual, 
constructive,  and  STOW  environments.  The  standardized  products 
generated  in  or  across  these  "environments  will  be  identical,  or 
nearly  identical,  and  appropriate  for  each  echelon  receiving  an 
AAR."  A  review  of  the  STAARS  ORD  provides  clarification  of 
the  concept  of  standardized  AAR  aids . 

Insures  a  Minimum  Set  of  AAR  Aids  Can  be  Automatically  Provided 
to  Support  AARs  Across  Environments 

One  should  not  interpret  the  STAARS  ORD  as  limiting  the  AAR 
products  that  can  be  used  by  a  particular  unit  type  or  echelon. 
Instead,  standardization  of  products  is  intended  to  make  sure 
that  a  certain  minimum  set  of  AAR  products  can  be  made  available 
while  minimizing  the  effort  required  to  prepare  these  aids. 

STAARS  includes  a  three-tier  concept.  Tier  I  will  use  an  expert 
system  to  "provide  automated,  standardized  AAR  products  connected 
to  the  commander's  Mission  Essential  Task  List  (METL)  for 
training  events  and  Exercise  Objectives."  Tier  II  will  include 
additional  AAR  products  that  can  be  selected  from  a  menu,  and 
Tier  III  will  give  users  the  ability  to  create  custom  AAR 
products  and  enable  these  new  products  to  be  added  to  the  Tier  I 
and  II  implementations. 

Requirement  for  Standardized  AAR  Aid  to  Fit  Every  Environment  is 
not  Absolute 


Standardized  AAR  products  are  not  limited  to  those  that  are 
supported  by  all  training  environments.  The  STAARS  ORD  points 
out  that  standardized  displays  may  be  modified,  or  new 
standardized  products  created  "to  take  advantage  of  the  superior 
ability  of  a  particular  LVC  or  STOW  environment  to  support 
training  or  evaluation  of  certain  types  of  training  objectives." 

Various  environments  offer  distinct  advantages  in  terms  of 
their  ability  to  provide  an  opportunity  for  practicing  particular 
collective  tasks  and/or  to  provide  precise  feedback.  For 
example,  the  virtual  environment  offers  certain  advantages  over 
the  live  force-on-force  environment  in  training  fire  distribution 
and  control  tasks,  because  an  electronic  battlefield  makes  it 
possible  to  know  where  every  round  impacts.  Given  the  location 
of  round  impacts,  it  is  possible  to  construct  a  fire  fight  aid 
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like  Figure  8,  showing  where  the  fires  of  two  platoons  are  being 
distributed. 

There  is  a  second  reason  for  capitalizing  on  the  strengths  of 
specific  environments  by  providing  aids  that  are  currently 
supported  by  only  one  or  two  environments.  Future  exercises  may 
involve  innovative  blending  of  the  three  environments  to  increase 
the  effectiveness  and  efficiency  of  training.  For  example, 

Figure  8  came  from  a  recent  study  suggesting  how  the  use  of  an 
engagement  model,  based  upon  virtual  algorithms,  running 
currently  with  force-on-force  live  engagements  can  be  used  to 
enhance  the  quality  of  information  available  to  support  AARs 
(Brown,  Nordyke,  Gerlock,  Begley,  and  Meliza,  1998) .  In  this 
case,  casualty  assessments  would  continue  to  be  based  upon 
engagements  simulated  with  the  Multiple  Integrated  Laser 
Engagement  Simulation  System  (MILES) ,  but  the  virtual  algorithms 
would  be  used  to  provide  information  about  where  rounds  that  miss 
the  target  impact.  To  enable  this  capability,  vehicles  would 
need  to  be  instrumented  to  provide  information  about  the  gun  tube 
azimuth,  gun  tube  elevation,  and  type  of  ammunition  fired  for 
each  firing  event. 


Legend 


Fully  Operational  ^Catastrophic  Loss  ^  Fire  Control  Kill  ^Mobility  Kill  ^  Commo  Kill 
No.  of  TOW  and  Tank  Main  Gun  Rounds  Fired  at  Dead  Vehicle 


Figure  8.  Percentage  of  friendly  fires  falling  with  various 
footprints,  showing  portions  of  the  enemy  force  not  engaged. 
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Standardization  Includes  Defining  the  Tactical  Events  Marking  or 
Bounding  the  Period  Covered  by  Each  Aid 

STAARS  will  "automate  the  production  and  nomination  of 
candidate  AAR  aids  by  providing  expert  logic  aids  for  linking 
exercise  objectives  to  data  collection  and  allow  the  trainer  to 
select  aids  relevant  to  a  specific  exercise"  (National  Simulation 
Center,  1997a) .  For  a  system  to  automatically  generate  AAR  aids, 
it  must  know  what  type  of  aid  to  produce  and  the  time  period  to 
be  covered.  In  a  few  cases  the  time  period  covered  will  be 
bounded  by  the  start  of  the  exercise  (STARTEX)  and  the  end  of  the 
exercise  (ENDEX) ,  but  in  most  cases  periods  will  be  bounded  by 
tactical  events.  For  example,  a  video  clip  from  the  enemy's 
perspective  can  begin  when  a  moving  platoon  first  has  LOS  with 
the  enemy  and  end  when  the  first  shot  is  fired.  Such  an  aid 
would  provide  a  useful  illustration  in  cases  where  the  platoon 
was  using  a  movement  technique  that  made  it  especially  vulnerable 
to  the  enemy  and  resulted  in  rapid  destruction  of  the  platoon  by 
enemy  fires.  In  a  case  where  the  video  clip  did  not  illustrate  a 
significant  aspect  of  performance  (e.g.,  the  platoon  moved  to  a 
battle  position  as  soon  as  it  established  LOS  with  the  enemy  and 
twenty  minutes  passed  before  either  side  opened  fire) ,  the 
trainer  would  not  be  required  to  use  the  clip  for  an  AAR. 

The  job  of  defining  the  aids  to  be  automatically  generated  by 
an  exercise  is  made  somewhat  complex  by  the  fact  that  no  one 
knows  what  the  nature  and  timing  of  tactical  events  will  be  until 
a  particular  exercise  is  over.  Consider  a  case  where  one  is 
designing  an  AAR  aid  with  the  objective  of  illustrating  how  well 
or  poorly  a  moving  platoon  reacts  to  enemy  contact.  The  types  of 
behaviors  of  interest  include  reporting  to  higher  headquarters, 
using  fire  to  destroy  and  suppress  the  enemy,  and  use  of  cover 
and  concealment.  A  video  clip  or  3D  view  with  synchronized 
replay  of  radio  communications  can  be  used  to  illustrate  these 
behaviors  (or  their  absence) ,  but  what  tactical  events  should 
decide  when  the  replay  begins?  "Contact"  with  the  enemy  may  be 
defined  as:  the  first  enemy  direct  fire  against  the  platoon;  the 
first  platoon  direct  fire  against  the  enemy;  the  platoon 
receiving  enemy  indirect  fire;  the  platoon  reporting  contact  to 
higher  headquarters;  a  crew  reporting  contact  on  the  platoon 
radio  net;  the  platoon  leader  ordering  a  contact  drill  or  action 
drill;  or  visual  contact  is  established  between  the  platoon  and 
the  enemy  force.  If  the  designer  of  the  aid  chooses  the  platoon 
reporting  contact  as  the  trigger,  and  the  platoon  is  destroyed 
before  reporting  contact,  no  aid  will  be  produced.  Similar 
problems  exist  with  using  the  other  events  in  isolation  as 
triggers  for  starting  the  aid.  The  solution  is  to  tell  the 
system  to  start  the  aid  whenever  any  one  of  these  tactical  events 
first  occur  (Brown,  Wilkinson,  Nordyke,  Riede,  Robideaux,  and 
Huysson,  1995) . 
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Standardization  Involves  Identifying  Relevant  Mission,  Enemy, 
Terrain,  Time,  and  Troop  (METT-T)  Variables 

This  aspect  of  standardization  is  concerned  with  deciding 
what  information  needs  to  be  included  in  specific  types  of  aids, 
and  it  is  concerned  with  identifying  changes  in  the  tactical 
situation  that  need  to  be  automatically  recorded  by  the  system  or 
recorded  by  an  observer.  For  example,  adding  LOS  displays  to  a 
plan  view  can  make  it  possible  to  decide  which  friendly  vehicles 
have  LOS  with  which  enemy  vehicles  at  a  specific  point  in  time. 
Alternatively,  a  system  can  be  programmed  to  automatically  assess 
and  record  when  any  friendly  vehicle  first  has  LOS  with  an  enemy 
vehicle. 

Applying  standards  to  measure  the  performance . of  collective 
tasks  requires  attention  to  the  specific  METT-T  situation  under 
which  a  task  is  performed  (Kerins,  Atwood,  and  Root,  1990) .  If 
one  wants  to  assess  the  quality  of  a  platoon's  movement  or  the 
adequacy  of  battle  positions  it  is  difficult  to  do  so  without 
having  information  about  the  terrain  situation  and  the  suspected 
location  of  enemy  forces .  This  information  about  the  METT-T 
situation  is  used  to  decide  when  certain  courses  of  action  are 
required  on  the  part  of  a  unit,  and  they  are  used  to  decide  if 
the  course  of  action  has  been  correctly  executed  (Did  the  unit's 
actions  successfully  influence  the  situation?) . 

The  importance  of  the  METT-T  situation  in  determining  which 
AAR  aids  are  relevant  to  an  exercise  can  be  illustrated  by  a 
recent  attempt  to  describe  how  the  diagnosis  of  unit  performance 
might  be  automated  in  the  Close  Combat  Tactical  Trainer  (CCTT) 
environment.  McCarthey  (1997)  described  how  the  electronic  data 
stream  from  the  exercise  could  be  used  to  decide  what  courses  of 
action  were  appropriate  and  measure  how  well  actions  were 
performed  for  an  exercise  in  which  one  mechanized  infantry 
platoon  and  one  tank  platoon  provide  supporting  fires  for  the 
assault  of  a  third  platoon.  A  review  of  this  work  shows  that 
there  are  at  least  thirty  different  METT-T  conditions  that 
influence  what  a  unit  should  do  and/or  or  provide  a  means  of 
measuring  the  quality  of  unit  performance  (e.g.,  after 
repositioning  "the  support  by  fire  force  can  identify  more 
vehicles  than  previously" ) . 
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Standardization  Can  Also  Apply  to  Aids  Providing  "How  to  Fight, 
and  "How  to  Train"  Information 


Standardization  of  AAR  aids  is  not  necessarily  limited  to 
those  aids  based  upon  events  from  the  exercise  proceeding  the 
AAR.  Standardized  products  may  include  aids  that  present 
alternate  courses  of  action,  such  as  excerpts  from  tactical 
doctrine,  magazine  articles,  and  aids  showing  data  from 
successful  exercises. 

"How  to  Fight"  AAR  aids  are  already  being  used  in 
standardized  fashion  within  the  live,  virtual,  and  constructive 
environments.  In  recent  visits  to  the  Army's  Combat  Maneuver 
Training  Center,  Joint  Readiness  Training  Center,  and  National 
Training  Center  the  author  found  that  observer/controllers  (0/C) 
and  analysts  maintained  electronic  and  paper-based  libraries  of 
AAR  aids  used  to  describe  "how  to  fight"  information  (Brown  et 
al.,  1998).  OCs  selected  the  aids  for  a  particular  exercise 
based  upon  their  diagnosis. 

The  author  also  observed  paper-based  libraries  of  such  aids 
being  used  for  AARs  after  simulation  networking  (SIMNET) 
exercises .  A  problem  noted  by  trainers  in  reviewing  an  AAR 
system  for  this  environment  was  that  it  did  not  provide  a  means 
of  displaying  "how  to  fight"  aids  (Meliza  and  Brown,  1996)  . 
Demonstrations  of  the  constructive  Corps  Battle  Simulation  (CBS) 
and  Vision  XXI  AAR  systems  at  AAR  conferences  have  shown  that 
these  systems  include  the  capability  to  store  and  display 
libraries  of  "how  to  fight"  data. 

The  STAARS  ORD  states  that  the  AAR  system  should  "provide  the 
capability  to  compare  unit  performance  to  doctrinal  standards  and 
lessons  learned  from  other  training  events . "  The  STAARS  ORD  also 
states  that  the  system  should  allow  the  user  to  "modify  briefing 
slide  templates;  access  and  integrate  portions  of  previous  AARs; 
and  add  modify  or  delete  libraries  of  graph,  test  and  figure 
displays  and  create  custom  displays." 
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Functions  of  AAR  Aids 

and  Types  of  Aids  Supporting  Each  Function 

All  AAR  aids  can  serve  a  number  of  functions,  but  certain 
types  of  aids  do  a  better  job  than  others  of  supporting  a 
specific  function.  The  major  functions  of  AAR  aids  and  the  types 
of  aids  supporting  each  function  are  described  below. 

Help  AAR  Leader  Diagnose  Unit's  Performance  Problems  in 
Preparation  for  the  AAR 

AAR  aids  can  be  used  by  an  AAR  leader  prior  to  the  conduct  of 
an  AAR  to  diagnose  the  strengths  and  weaknesses  of  a  particular 
unit.  These  aids  are  used  to  supplement  any  information  the 
leader  may  have  gained  by  observing  the  unit's  performance  during 
an  exercise. 

The  goal  of  the  AAR  leader  is  to  go  beyond  identifying 
problems  in  the  exercise  outcome  to  the  point  of  finding  out  what 
caused  the  outcomes .  The  AAR  leader  should  also  be  concerned 
with  defining  the  link  between  outcomes  and  causes  so  that  the . 
unit  will  be  motivated  to  correct  the  problem.  In  defining  this 
link,  the  leader  diagnoses  performance  in  increasing  levels  of 
detail.  For  example,  a  company  team  failed  to  take  an  objective, 
because  two  platoons  were  destroyed  by  enemy  direct  fire.  The 
assault  element  was  destroyed,  because  the  artillery  fire 
intended  to  suppress  the  enemy  was  not  at  the  correct  location. 
The  fire  was  at  the  wrong  location,  because  there  was  no  attempt 
to  adjust  fires. 

The  types  of  aids  useful  for  post-exercise  diagnosis  are 
those  that  can  be  quickly  screened  to  identify  problems  in  unit 
performance  such  as  data  tables,  graphs,  exercise  timelines, 
battle  flow  displays,  fire  fight  displays,  and  range  fans .  The 
term  range  fan  is  used  here  to  cover  a  variety  of  utilities  that 
have  been  developed  to  show  how  well  the  positions  occupied  by 
individual  war fighting  components  enable  the  unit  to  observe  and 
fire  on  the  area  of  operations. 

One  of  the  most  valuable  aids  for  diagnosing  performance  are 
the  checklists  used  by  trainers  in  observing  the  performance^  of 
key  leaders.  One  of  these  lists,  for  example,  can  address  the 
actions  a  platoon  leader  should  take  when  preparing  a  platoon  for 
a  mission  and  include  such  actions  as  asking  questions  of 
subordinates  to  make  sure  they  understand  the  mission.  In  many 
cases,  a  trainer  may  anticipate  outcome  problems  due,  for 
example,  to  a  unit  leader's  failure  to  designate  responsibility 
for  performing  a  task  to  a  specific  crew  or  individual.  If  these 
outcome  problems  subsequently  occur,  the  AAR  leader  will  have 
already  identified  a  chain  of  causality. 
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Illustrate  or  Document  Performance  Problems 


The  AAR  process  begins  to  fall  apart  if  units  do  not 
recognize  that  performance  problems  exist,  and  an  AAR  leader's 
comments  about  unit  performance  (e.g.f  "the  platoon  did  not  move 
fast  enough")  are  often  not  sufficient  to  cause  a  unit  to 
acknowledge  a  problem.  If  an  AAR  leader  can  illustrate  or 
document  a  problem  (e.g.,  "in  this  replay  you  can  see  that  you 
crossed  Phase  Line  Zebra  at  0612,  and  at  what  time  were  you 
supposed  to  cross  Zebra?"),  a  unit  is  more  likely  to  accept  that 
a  problem  exists. 

The  documentation  function  can  be  served  by  2D  replays,  video 
clips,  3D  replays,  data  tables,  graphs,  timelines  of  exercise 
events,  battle  flow  displays,  fire  fight  displays,  replays  of 
radio  communications  (perhaps  synchronized  with  2D  or  3D  replays 
or  video  clips),  and  range  fans. 

Refresh  Memories  of  Exercise  Participants 

AAR  aids  can  be  used  to  refresh  the  memories  of  exercise 
participants  by  recreating  the  situation  experienced  by  unit 
members  during  an  exercise  (e.g.,  "this  is  the  terrain  situation 
faced  by  your  moving  platoon  when  it  first  received  enemy 
fires") .  The  aids  used  for  this  purpose  should  match  what  the 
unit  saw  and  heard  during  the  exercise.  The  aids  supporting  this 
function  include  video  clips  showing  activities  from  the 
perspective  of  the  unit  being  trained  and  3D  replays  of  exercise 
data  appropriate  to  the  virtual  format.  Replays  of  tactical 
radio  communications  (perhaps  synchronized  with  video  clips  and 
3D  replays)  can  also  be  used  to  refresh  the  memories  of  exercise 
participants.  Finally,  descriptions  of  the  unit's  orders  or 
planning  documents  can  help  to  refresh  memories  of  participants 
regarding  the  unit's  goals  and  planned  approach. 

Provide  New  Perspectives  on  Exercise  Events 

AAR  aids  can  also  be  used  to  provide  a  new  perspective 
regarding  what  happened  during  an  exercise  (Meliza,  1997) .  An 
example  of  providing  new  perspectives  would  be  using  a  replay  to 
show  a  platoon  that  the  "enemy  force"  they  briefly  and 
unsuccessfully  engaged  was  the  rear  element  of  a  friendly  company 
team.  The  types  of  AAR  aids  supporting  this  function  include  two 
dimensional  replays  from  a  plan  view,  video  clips  or  3D  replays 
from  the  perspective  of  the  enemy,  data  tables,  graphs,  timelines 
of  exercise  events,  battle  flow  displays,  fire  fight  displays, 
replays  of  radio  communications,  and  range  fans.  That  is,  all  of 
the  aids  for  providing  ground  truth  data  can  be  used  to  provide 
new  perspectives. 
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Help  Units  Examine  Alternative  Courses  of  Action 


AAR  aids  can  also  be  used  to  provide  information  from  outside 
a  particular  exercise  that  helps  to  diagnose  problems  in  unit 
performance  or  point  out  corrective  actions .  "How  to  fight  and 
"how  to  train"  AAR  aids  address  this  function.  Using  the  AAR 
process  to  help  a  unit  identify  action  items  necessary  to  improve 
or  sustain  performance  is  a  challenging  responsibility.  Recent 
research  by  Gubler  (1997)  found  that  very  few  AARs  produced 
specific  training  solutions  to  identified  problems  in  unit 
performance.  Clearly,  additional  research  is  needed  to  decide 
how  to  best  use  AAR  aids  to  address  this  need. 


Encourage  Participation  in  the  AAR  and  Press  Unit  Members 
to  Identify  Problem  Causes  and  Potential  Solutions 

The  AAR  process  simply  does  not  work  unless  unit  members, 
participate  in  discussions.  One  of  the  most  commonly  used  aids 
for  soliciting  participation  in  the  AAR  are  AAR  orientation, 
slides  used  at  the  beginning  of  the  AAR.  Such  slides  may  simply 
state  that  "this  is  your  review  of  your  unit's  performance  and  I 
am  here  to  support  your  review  process." 

The  next  most  commonly  used  aids  is  probably  a  map  display  of 
the  area  in  which  the  exercise  was  conducted  containing  the 
unit's  control  measures.  The  AAR  leader  often  asks  the  unit 
leader  to  start  the  AAR  by  using  the  display  to  describe  the 
unit's  plan  for  the  operation.  Another  commonly  used. type  of  aid 
is  created  on  the  spot  when  an  AAR  Leader  creates  a  list  of 
responses  to  questions  such  as  "what  issues  do  you  want  to  cover 
during  your  AAR"  or  "what  do  you  think  was  your  unit's  greatest 
weakness  (or  strength)  during  this  exercise."  Another  commonly 
used  aid  is  a  model  of  the  terrain  on  which  the  exercise  was 
conducted  with  "toy"  vehicles  that  unit  members  can  use  to  show 
what  happened  in  the  exercise. 

Motivating  and  cueing  unit  participation  in  an  AAR  is  a 
difficult  task.  A  recent  review  of  platoon  and  company  level  AAR 
tapes  from  maneuver  combat  training  centers  showed  large 
differences  among  AAR  sessions  in  terms  of:  the  relative  amount 
of  talking  done  by  the  AAR  leader  versus  the  unit;  the  number  of 
problems  or  corrective  solutions  identified  by  the  AAR  leader 
versus  those  identified  by  the  unit;  and  the  percentage  of  the 
unit  that  participated  in  the  AAR  (Gubler,  1997).  Only  four  of 
the  seventeen  sessions  examined  demonstrated  a  high. level  of 
participation  by  unit  members.  This  research  also  involved 
looking  for  relationships  between  the  actions  of  AAR  leaders  and 
the  level  of  participation.  One  AAR-aid  related  factor 
associated  with  a  high  level  of  participation  was  the  use. of 
battle  damage  assessment  (BDA)  data.  Unfortunately  we  still  know 
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very  little  about  how  specific  types  of  AAR  aids  influence  the 
level  of  participation  of  unit  members . 

In  certain  cases  unit  members  may  do  a  lot  of  talking  during 
the  AAR  without  identifying  causes  of  problems  or  corrective 
actions.  One  of  the  techniques  used  to  push  unit  members  into 
identifying  specific  causes  and  corrective  actions  is  to  ask 
members  of  the  unit  such  questions  as  "if  you  were  just  starting 
the  last  exercise  what  would  you  do  differently  this  time" ,  and 
then  prepare  a  list  of  responses  for  all  to  view. 

Summarize  and  Document  the  Results  of  the  AAR 

The  summary  of  the  findings  during  the  AAR  may  be  provided  by 
the  AAR  leader  or  they  may  be  provided  by  unit  members  in 
response  to  questions  from  the  AAR  leader  such  as  "what  do  you 
think  your  highest  priority  action  items  should  be  in  preparing 
for  future  exercises?"  In  many  cases  the  AAR  leader  may  know 
what  he  wants  the  unit  to  discover  during  the  AAR  and  prepare 
appropriate  slides  before  the  start  of  the  AAR. 


Table  1  provides  a  summary  of  AAR  aid  functions  and  types  of 
aids  associated  with  each  function. 
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Table  1. 


Major  Functions  of  After  Action  Review  (AAR)  Aids  and  Types  of 
Aids  Supporting  Each  Function 


Functions 

Types  of  Aids 

Help  AAR  leader  diagnose 
problems  in  preparation 
for  the  AAR 

Behavioral  checklists,  tables, 
graphs,  battle  flow  displays, 
fire  fight  displays,  range 
fans,  exercise  timelines. 

Refresh  memories  of 
exercise  participants 

3D  replays  and  video  clips 
from  unit's  perspective, 
radio  communications, 
terrain  models,  unit  plans 

Provide  new  perspectives 
on  exercise  events 

3D  replays  and  video  clips  from 
enemy's  perspective,  2D  replays, 
tables,  graphs,  battle  flow  & 
fire  fight  displays,  range  fans, 
radio  communications 
synchronized  with  2d  replays 
exercise  timelines, 
behavioral  checklists 

Illustrate  or  document 
performance  problems 

2D  or  3D  replays,  video  clips, 
tables,  graphs,  battle  flow  & 
fire  fight  displays,  range 
fans,  exercise  timelines 
radio  communications 
(synchronized  with  replays) 

Help  unit  members  examine 
alternative  courses  of 
action 

Behavioral  checklists; 
excerpts  from  tactical 
doctrine,  journals,  magazines; 
Illustrations  of  performance 
from  successful  exercises; 

Lists  generated  by  participants 

Encourage  participation 
and  push  unit  to  identify 
specific  problem  causes 
and  potential  solutions 

Lists  generated  by  participants, 
terrain  models,  3D  sand  tables; 
tables  showing  battle  damage 
assessments 

Summarize  and  document 
results  of  the  AAR 

Lists  generated  by  participant 
input  or  by  leader 
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Techniques  for  Defining  a  Standardized  Set 
of  AAR  Aids  for  a  Specific  Unit  Type  and  Echelon 

One  of  the  biggest  obstacles  to  having  proponents  define 
standardized  AAR  products  for  specific  unit  types  and  echelons  is 
that  few  units  have  significant  experience  using  AAR  aids.  Even 
at  the  maneuver  CTCs,  for  example,  the  AAR  aids  available  at. 
platoon  and  company  level  are  largely  of  the  type  that  describe 
alternative  courses  of  action  (e.g.,  excerpts  from  manuals) 
rather  than  the  type  that  provide  information  about  the  actions 
of  the  unit  during  an  exercise.  At  present  we  lack  the  personnel 
resources  and  automation  capabilities  needed  to  provide  exercise- 
based  AAR  aids  in  a  timely  fashion.  When  we  consider  the  virtual 
world  represented  by  SIMNET  or  CCTT,  we  find  that  the  typical  AAR 
aid  is  an  exercise  replay  from  a  3D  or  2D  perspective.  Even 
basic  statistical  displays  showing  who  shot  whom  are  rare. 

The  STAARS  AAR  Handbook  (National  Simulation  Center,  1997b) 
provides  examples  of  AAR  products  based  on  those  used  currently 
for  CBS  and  Brigade  Battle  Simulation  (BBS)  AARs .  In  addition  to 
describing  and  illustrating  each  aid,  the  handbook  describes  how 
the  aid  would  be  used,  describes  the  general  data  sources  used  to 
prepare  the  aid,  and  identifies  echelons  for  which  the  aids  would 
be  appropriate  (e.g.,  Brigade  and  higher).  One  approach  to 
defining  the  standardized  AAR  products  for  a  specific  echelon  and 
unit  type  is  to  begin  with  examples  in  the  handbook  and  decide 
how  the  aid  can  be  tailored  to  fit  your  unit. 

This  current  report  suggests  an  alternative  approach  focusing 
on  the  tasks  to  be  trained  by  specific  unit  types  and  echelons. 

It  is  less  sensitive  to  the  effects  of  lack  of  experience  with 
AAR  aids ,  because  the  information  requirements  are  developed 
before  the  issue  of  specific  AAR  aid  types  arises. 

Step  One:  Decide  what  actions  needs  to  be  illustrated  to  support 
application  of  specific  task  standards 

The  first  step  in  this  process  is  to  describe  the  unit 
actions  to  be  illustrated  by  AAR  aids .  These  actions  may  be 
stated  in  a  positive  or  negative  manner.  For  example,  the . 
actions  may  describe  what  the  unit  should  do  (the  unit  arrives  at 
control  measures  on  time) ,  or  they  may  describe  problems  in 
performance  (unit  movement  is  slowed  by  frequent  halts) . 

Mission  Training  Plan  documents  are  often  a  good  source  of 
information  about  positive  and  negative  actions  appropriate  to  a 
particular  unit.  The  examples  in  Table  2  from  the  Mission 
Training  Plan  for  tank  platoons  (Department  of  the  Army,  1988) . 
Other  sources  of  information  include  reports  from  the  Center  for 
Army  Lessons  Learned. 
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Table  2. 


Actions  to  be  Illustrated  When  Applying  Task  Standards  for  Sample 
Armor  Platoon  Tasks 


Task/ Standard 


Actions 


Execute  a  Column  Formation/ 
The  platoon  executes  the 
column  formation  without 
delay  &  without  stopping 
movement 


Formation  change  takes  too 
long 

Vehicles  stop  moving 
when  changing  formations 

Collisions  or  near  collisions 


Formation  does  not  fit 
definition  of  column  in  terms 
of  spatial  relationships  & 
distances  _ 


Occupy  a  Platoon  Battle 
Position/  The  platoon 
moves  to  the  assigned  BP, 
completes  the  deliberate 
occupation,  and  reports 
established  at  or  before 
the  "defend  NLT"  time 
given  in  the  OPORD. 
Coordination  with  adjacent 
platoons  is  conducted, 
conflicts  are  resolved, 
and  changes  to  the 
platoon's  plan  resulting 
from  the  coordination  are 
disseminated . 


Unit  does  not  move  to  assigned 
BP 

Unit  does  not  report 
occupation  completed  by  NLT 
time 

Gaps  in  fields  of  fire  or 
observation  between  platoons 

Gaps  in  platoon' scoverage  of 
its  assigned  sector 

Positions  offer  poor  cover  and 
concealment 


Process  Enemy  Prisoners 
of  War/  The  platoon 
utilizes  the  five  "S's"  in 
processing  enemy  prisoners 
of  war  (EPWs )  for  evacuation 
to  the  next  higher 
headquarters '  EPW  holding 
area  within  6  hours 
of  capture. 


Unit  late  moving  EPW  to  rear 

Critical  information  is  not 
provided  by  reports  or 
tagging  of  EPWs  (number 
captured,  date  and  time, 
grid  coordinates,  & 
circumstances) 


EPW  are  not  segregated. 


Step  Two:  Identify  Markina  or  Bounding  Tactical  Events 
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The  second  step  in  identifying  a  standard  set  of  AAR  aids  is 
to  decide  what  tactical  events  mark  or  bound  the  period  covered 
by  displays.  In  certain  cases,  an  aid  will  be  based  upon  the 
situation  that  exists  at  a  specific  point  time  associated  with 
the  occurrence  (or  marking)  of  a  tactical  event.  In  other  cases, 
the  period  will  cover  a  span  of  time  and  be  bounded  by  the 
occurrence  of  two  tactical  events .  Table  3  shows  tactical  events 
that  can  be  used  to  mark  or  bound  the  time  periods  relevant  to 
the  actions  shown  in  Table  2 . 

Consideration  of  marking  and  bounding  events  helps  in 
identifying  the  types  of  aids  appropriate  to  illustrating  a 
particular  unit  action.  Certain  aids  are  appropriate  only  when  a 
span  of  time  is  addressed  (e.g.,  a  battle  flow  display),  some  are 
appropriate  only  for  describing  the  situation  at  a  specific  point 
(e.g.  snapshot) ,  and  others  can  be  applied  to  support  either 
situation.  Table  3  provides  examples  of  both  marking  (specific 
point  in  time)  and  bounding  (time  span)  events. 

This  step  also  sets  the  stage  for  Tier  1  and  Tier  2  AAR 
systems  that  can  automatically  generate  AAR  aids  for  users . 
Marking  and  bounding  events  define  the  data  stream  used  to 
prepare  specific  aids.  For  example,  the  purpose  of  a  display  may 
be  to  illustrate  that  a  unit  promptly  reported  contact  to  the 
next  higher  unit.  The  period  of  time  to  be  covered  by  the 
display  begins  when  contact  is  made  and  ends-  when  contact  has 
been  reported.  If  the  purpose  of  the  display  is  to  show  that  a 
unit  did  not  report  contact  promptly,  the  period  to  the  covered 
begins  with  contact  and  ends  when  a  contact  report  would  no 
longer  be  considered  prompt  (e.g.,  two  minutes  after  contact) . 
Fortunately,  the  marking  and  bounding  tactical  events  usually  do 
not  differ  between  a  positively  stated  action  and  a  negatively 
stated  action. 

Step  Three:  Identify  Important  Situation  Variables 

The  third  step  in  defining  standardized  aids  is  to  identify 
aspects  of  the  METT-T  situation  that  need  to  be  examined  when 
applying  task  standards.  This  information  helps  to  define  the 
general  type  of  AAR  aid  needed  and  the  specific  design  features 
needed  to  examine  unit  behavior.  For  example,  if  one  is  trying 
to  decide  if  vehicles  moving  in  a  platoon  maintain  appropriate 
spacing  it  is  important  to  know  the  terrain  situation.  The 
display  must  provide  information  about  the  spatial  distribution 
of  vehicles  and  the  terrain  situation.  As  previously  mentioned . 
in  this  guide  the  generic  types  of  AAR  aids  can  be  implemented  in 
many  ways,  depending  upon  the  design  features  needed  to 
illustrate  the  actions  of  interest.  Table  4  provides  examples  of 
METT-T  variables  that  need  to  be  considered  when  examining  unit 
performance . 
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Table  3 . 


Tactical  Events  Marking  or  Bounding  Actions  to  be  Illustrated  by 
AAR  Aids 


Actions 


Boundaries 


Execute  a  Column  Formation 

Change  formations  too  slowly  Unit  ordered  to  move  in 

column  formation  until 
15  seconds  after  order 


Vehicles  stop  moving  when 
changing  formations 

Collisions  or  near  collisions 


Formation  does  not  fit 
definition  of  column  in 
terms  of  spatial  relation¬ 
ships  and  distances 


15  seconds  after  order 
to  move  in  column  until 
formation  change 
ordered  or  contact  made 


Occupy  a  platoon  BP 

Unit  does  not  move  to  assigned  BP  NLT  time' 


Unit  does  not  report  occupation 
completed  by  NLT  time 

Gaps  in  fields  of  fire  or 
observations  between  platoons 

Gaps  in  the  platoon's  coverage 
of  its  assigned  sector 

Positions  offer  poor  cover  & 
concealment 


Unit  reports  occupation 


NLT  time 


NLT  time 


NLT  i-ime 


Process  EPW 

Platoon  late  moving  EPW  to  rear 


Critical  information  is 
not  provided  by  reports 
or  tagging  of  EPWs 

EPW  are  not  segregated 


EPW  arrive  at  rear 
holding  area 

Time  EPW  arrive  at  rear 
holding  area  or  platoon 
reports  EPW  situation 

1st  EPW  captured  &  EPW 
reach  rear  holding  area 
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Table  4 . 


Situation  Variables  Important  in  Reviewing  Unit  Actions 
_ Actions _ _ _ Variables 

Execute  a  column  formation 

Formation  change  takes  too  long 

Vehicles  stop  moving  when  changing 
formations 

Collisions  or  near  collisions 

Turn  flanks  towards  suspected 
enemy  locations 

Formation  does  not  fit  definition 
of  column  in  terms  of  spatial 
relationships  and  distances 

Occupy  a  platoon  BP 

The  platoon  does  not  move 
to  the  assigned  BP 

Unit  does  not  report  occupation 
completed  by  NLT  time 

Gaps  in  fields  of  fire  or 
observation  between  platoons 


Liire, 

Limits  of  platoon's  BP, 
platoon's  sector  of 
fire,  terrain 

Positions  offer  poor  Terrain 

cover  and  concealment 

Process  EPW 

Late  moving  EPW  to  rear  Time  EPW  are  captured 

Critical  information  is  not 
provided  by  reports 
or  EPW  tagging  (number 
captured,  date  and  time) 

EPW  are  not  segregated.  _ _ ; _ _ _ 


Gaps  in  platoon's  coverage 
of  its  assigned  sector 


Terrain 

Terrain 

Terrain 

Terrain  &  Suspected 
Enemy  Locations 

Terrain 

Location  of  assigned 
BP 

NLT  time 

Limits  of  platoon's  BP 
platoon's  sector  of 
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Step  Four:  Estimate  Display  Requirement 


The  job  of  describing  display  requirements  is  largely 
complete  by  the  time  proponents  finish  step  three.  Step  four 
provides  proponents  with  the  opportunity  to  specify  a  type  of  aid 
or  types  of  aids  appropriate  to  applying  a  specific  aspect  of  a 
task  standard.  Step  four  is  where  a  proponent  may  describe  an 
idea  for  a  new  type  of  display,  a  new  graph,  a  new  data  table,  or 
the  addition  of  new  features  to  existing  types  of  displays. 

It  is  important  to  consider  that  more  than  one  type  of  aid 
may  be  of  use  in  examining  a  particular  aspect  of  unit  behavior. 
In  such  cases  it  is  useful  to  identify  the  different  types  of  AAR 
aids  that  would  be  appropriate .  There  are  many  cases  in  Table  5 , 
for  example,  where  either  a  2D  or  3D  replay  can  be  used. 
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Table  5 . 


Display  Requirements  for  Illustrating  Actions 

Actions  _ _ _ Display 

Execute  a  Column  Formation 


Formation  change  takes  too  long 


2D/3D  replay  or  video 


Vehicles  stop  when  changing 
formations 


2D,  3D  replay  +  show 
vehicle  speeds 


Collisions  or  near  collisions 


2D/3D  replay  or  video 


Turning  flanks  towards  suspected 
enemy  locations 


"  "  (show  enemy  with 
2D  &  enemy  perspective 
for  3D  and  video 


Formation  does  not  fit  definition  2D 
of  column  in  terms  of  spatial 
relationships  and  distances 


Occupy  a  BP 

Unit  does  not  move  to  assigned  BP  2D,  3D  'or  video  clip 


Unit  does  not  report  occupation 
completed  by  NLT  time 


Timeline  showing  NLT 
and  report  time  or  time- 
tagged  replay  of  report 


Gaps  in  fields  of  fire  or 
observation  between  platoons 
& 

Gaps  in  platoon's  coverage  of 
its  assigned  sector 

Positions  offer  poor  cover  and 
concealment 

Process  EPW 

Late  moving  EPW  to  rear 


Range  fan  showing 
areas  covered  by 
observation  and  fire 


3D  or  video  clip  from 
enemy ' s  perspective 


Timeline  showing  when 
EPW  were  captured  and 
when  EPW  arrived  at 
rear  holding  area 


Critical  information  is  not  Replay  of  reports  or 

provided  by  reports  or  EPW  tagging  EPW  tags _ 
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The  Role  of  a  Flexible  AAR  Aid  Editing  System 

If  AAR  systems  allow  users  to  change  AAR  aids  to  be . 
automatically  produced  and  also  make  last  minute  edits  in  aids, 
then  proponents  need  not  worry  about  having  a  perfect  set  of 
standard  aids  prior  to  initial  fielding  of  these  systems.  Both 
capabilities  are  within  the  state  of  the  art  of  AAR  systems,  but 
proponents  must  perform  certain  analytical  tasks  early  to  ensure 
implementation  of  these  capabilities. 

Capability  to  Edit  the  Time  Addressed  by  AAR  Aids 

There  will  be  times  when  automatically  generated  aids  do  not 
meet  user  needs  in  terms  of  the  time  addressed,  such  as  when  a 
significant  event  occurs  immediately  beyond  the  period  covered  by 
an  aid.  We  need  to  provide  users  with  the  ability  to  change  the 
point  or  span  of  time  addressed  by  aids.  The  Automated  Training 
Analysis  and  Feedback  System  (ATAFS) ,  for  example,  gives  users 
the  capability  to  edit  AAR  aids,  including  the  ability  to  change 
time  (Brown,  Wilkinson,  Nordyke,  Hawkins,  Robideaux,  and 
Huyssoon,  1996;  Brown,  Wilkinson,  Nordyke,  Hawkins,  Riede, 
Robideaux,  and  Huysson,  1996) .  In  the  example  in  Figure  9  a  user 
could  change  the  end  time  to  include  an  event  that  occurred  after 
the  contact  report.  ATAFS  is  able  to  support  editing  of  time, 
because  it  saves  definitions  of  aids  that  it  can  apply  to  a  log 
file  of  exercise  data  rather  than  saving  the  aid  per  se. 


I  Map 


Display 


Help 


Actions  of  Tank  Cdr  Initiating  the  Action 


Start  Time: I  00:05:11  :  Ten  seconds  prior  to  contact 


End  Time:  l  00^05:23  |  j  Contact  report  transmitted^ 


1st  Pit,  Co  A,  3-70  AR 
18  Nov  94 
Exercise  012 
Movement  to  Contact 


Key  points  for 
AAR  discussion: 


1.  How  well  did  the  ( 
tank  cdr  initiating  the  | 
action  return  or  initiate! 
fires? 

2.  Did  he  seek  cover 
and  concealment? 

3.  What  did  he  report? 

4.  How  well  did  his 
perception  of  the 
threat  compare  to  the 
actual  threat? 

'  ’■  *• " "  "  ~ 


;  a  |  m  r>l 


MM&ML  \  £ 


Figure  9.  AAR  aid  editing  screen. 
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Capability  to  Implement  the  Automation  of  New  AAR  Aids  or  Modify 
Existing  Aids 


The  STAARS  interface  that  will  allow  proponents  to  change 
Tier  I  products  has  yet  to  be  designed,  but  we  have  an  idea  how 
this  system  will  work  based  upon  similar  products.  Figures  10 
and  11  illustrate  an  interface  that  can  be  used  to  automate  the 
production  of  a  new  aid  (Brown,  Wilkinson,  Nordyke,  Riede, 
Huyssoon,  Aguilar,  Wonsewitz,  and  Meliza,  1997;  Brown,  Wilkinson, 
Noredyke,  Riede,  Robideaux,  and  Huysson,  1995)  .  In  this  example, 
the  user  has  decided  to  automate  the  production  of  an  animated  2D 
display  covering  a  period  that  begins  when  a  platoon  crosses  its 
Line  of  Departure  (LD)  and  ending  when  enemy  contact  is  reported 
within  the  platoon.  The  user  selects  a  movie  camera  icon 
(representing  an  animated  2D  display)  to  specify  the  type  of  aid 
to  be  created.  The  user  selects  a  line  icon  and  positions  it 
next  to  a  green  light  to  specify  a  line  crossing  will  begin  the 
period  covered  by  the  aid.  He  or  she  then  selects  a  "prompt" 
icon  and  positions  it  next  to  the  stop  sign  to  specify  a 
trainer's  observation  that  an  event  has  occurred  (e.g.,  contact 
has  been  reported)  will  end  the  period  addressed  by  the  aid.  The 
user  can  then  give  the  aid  a  descriptive  name  and  name  the  events 
that  will  trigger  the  production  of  the  aid. 

Figure  11  shows  how  a  user  can  complete  the  process  of 
automating  the  production  of  an  aid  by  providing  more  specific 
descriptions  of  the  trigger  events.  In  this  case,  the  user  has 
selected  a  Phase  Line  from  a  list  of  lines  and  has  responded  to  a 
menu  that  allows  selection  of  the  criteria  for  assessing  when  a 
unit  has  crossed  the  LD  (i.e.,  when  the  first  vehicle,  center  of 
mass,  or  last  vehicle  crosses  the  line) . 

A  user  can  modify  these  standardized  AAR  aids  by  changing  the 
type  of  aid  to  be  created,  the  type  of  triggers  used  to  begin  and 
end  the  period  addressed  by  the  aid,  the  parameters  used  to 
define  triggers,  the  title  of  the  aid,  and  the  description  of  the 
tactical  events  associated  with  the  start  and  stop  points  of  the 
period  addressed  by  the  aid. 
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Title  1  Movement  from  LP  to  First  Enemy  Contact 


Event  |  Platoon  Crosses  the  LD 


gm  Event  !  Contact  Report  to  Platoon 


Ui»T*» 


Trigger-Line-UM-ast  Vehicle 


Figure  11.  Screen  for  selecting  criteria  for  deciding  when 
marking  or  bounding  events  triggering  AAR  aid  production  have 
occurred. 


What  Proponents  Should  Do  to  Support  the  Implementation  of 
Flexible  Systems 


There  are  two  sets  of  decisions  that  must  be  made  by 
proponents  prior  to  the  initial  fielding  of  a  STAARS . 

Proponents  must  decide  what  types  of  displays  will  be  used  for 
AARs,  and  they  must  identify  the  tactical  events  and  event 
parameters  likely  to  be  of  interest  to  the  unit  types  and 
echelons  for  which  they  have  proponency.  U.S.  Army  Simulation 
Training  and  Instrumentation  Command  engineers,  given  this 
information,  can  look  across  the  requirements  of  a  various  user 
groups  to  decide  whether  it  is  feasible  for  set  of  AAR  software 
to  apply  to  a  wide  variety  of  groups.  Information  about  the 
requirements  of  specific  user  groups  can  also  be  used  to 
implement  standardized  AAR  products  and  the  capability  for  users 
to  modify  standardized  AAR  aids. 
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Summary 


AAR  aids  can  be  used  to  help  trainers  diagnose  unit  strengths 
and  weaknesses  prior  to  the  AAR.  They  can  also  be  used  to 
refresh  memories  of  unit  members  regarding  exercise  events, 
provide  new  perspectives  regarding  exercise  events,  document 
problems  in  unit  performance,  describe  alternative  courses  of 
action  a  unit  can  take  in  the  future,  stimulate  unit 
participation  in  the  AAR  process,  and  document  the  results  of  an 
AAR  session. 

The  STAARS  concept  is  intended  to  facilitate  training 
feedback  across  the  live,  virtual  and  constructive  environments. 
Standardization  of  AAR  products  across  these  environments  for 
particular  unit  types  and  echelons  facilitates  the  development  of 
software  that  can  be  used  to  automate  the  AAR  aid  preparation 
process .  Standardization  makes  it  possible  to  examine  AAR  aid 
requirements  across  unit  types,  echelons,  and  environments  and 
envision  a  minimum  number  of  software  systems  that  can  apply 
across  training  situations,  reducing  the  costs  of  developing  and 
maintaining  AAR  software.  Standardization  also  helps  to  make 
sure  that  units  can  readily  interpret  the  training  points  the 
various  aids  attempt  to  make . 

The  process  of  standardizing  AAR  aids  is  made  difficult  by 
the  fact  that  most  units  have  very  limited  experience  with  the 
different  types  of  displays  that  can  be  used  to  support  AARs . 

This  report  describes  a  process  whereby  the  proponents  for 
specific  unit  types  and  echelons  can  define  standardized  AAR 
products.  This  process  involves  deciding  what  performance 
problems  a  proponent  may  want  to  illustrate  for  collective  tasks 
described  in  MTP  documents,  deciding  what  aspects  of  the  tactical 
situation  need  to  be  included  when  assessing  specific  problems, 
and  identifying  tactical  events  that  mark  or  bound  the  time 
period  to  be  covered  by  the  aid.  In  an  optional  fourth  step,  a 
proponent  may  even  go  so  far  as  to  specify  the  specific  aids  that 
may  be  used  to  illustrate  specific  behavior  problems  and  describe 
new  types  of  aids . 

By  defining  standardized  AAR  products  for  a  particular  unit 
type  and  echelon,  a  proponent  will  have  improved  input  for 
deciding  upon  AAR  system  requirements.  Further,  by  looking 
across  the  standardized  AAR  products  for  many  different  unit 
types,  the  U.S.  Army  may  find  that  it  can  reduce  the  total  number 
of  AAR  systems  that  need  to  be  developed  and  maintained  to  just  a 
few. 
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